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Abstract. Recent work by Hermanns et al. and Kattenbelt et al. has 
extended counterexample-guided abstraction refinement (CEGAR) to 
probabilistic programs. These approaches are limited to predicate ab- 
straction. We present a novel technique, based on the abstract reacha- 
bility tree recently introduced by Gulavani et al., that can use arbitrary 
abstract domains and widening operators (in the sense of Abstract In- 
terpretation). We show how suitable widening operators can deduce loop 
invariants difficult to find for predicate abstraction, and propose refine- 
ment techniques. 



1 Introduction 

Abstraction techniques are crucial for the automatic verification of systems 
with a finite but very large or infinite state space. The Abstract Interpre- 
tation framework provides the mathematical basis of abstraction [5]. Recent 
work has extended abstraction techniques to probabilistic systems using games 
|12ll3ll5l21l22j . The systems (e.g. probabilistic programs) are given semantics in 
terms of Markov Decision Processes (MDPs), which can model nondeterminism 
and (using interleaving semantics) concurrency. The key idea is to abstract the 
MDP into a stochastic 2-Player game, distinguishing between nondeterminism 
inherent to the system (modeled by the choices of Player 1) and nondeterminism 
introduced by the abstraction (modeled by Player 2). The construction ensures 
that the probability of reaching a goal state in the MDP using an optimal strat- 
egy is bounded from above and from below by the supremum and infimum of 
the probabilities of reaching the goal in the 2-Player game when Player 1 plays 
according to an optimal strategy (and Player 2 is free to play in any wayj^J. An 
analogous result holds for the probability of reaching a goal state in the MDP 
using a pessimal strategy. 

The abstraction technique of [15122) and the related [12113121] relies on pred- 
icate abstraction: an abstract state is an equivalence class of concrete states, 
where two concrete states are equivalent if they satisfy the same subset of a given 
set of predicates. The concretization of two distinct abstract states is always dis- 
joint (the disjointness property). If the upper and lower bounds obtained using a 
set of predicates are not close enough, the abstraction is refined by adding new 
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predicates with the help of interpolation, analogously to the well-known CEGAR 
approach for non-probabilistic systems. 

While predicate abstraction has proved very successful, it is known to have 
a number of shortcomings: potentially expensive equality and inclusion checks 
for abstract states, and "predicate explosion" . In the non-probabilistic case, the 
work of Gulavani et al. has extended the CEGAR approach to a broader range 
of abstract domains [llj . in which widening operations can be combined with 
interpolation methods, leading to more efficient abstraction algorithms. We show 
that the ideas of Gulavani et al. can also be applied to probabilistic systems, 
which extends the approaches of [15116122] to arbitrary abstract domains. Given 
a probabilistic program, an abstract domain and a widening for this domain, we 
show how to construct an abstract stochastic 2-Player reachability game. The 
disjointness property is not required. We prove that bounds on the probability 
of reaching a goal state in the MDP can be computed as in [15122) . The proofs 
of [22] use the disjointness property to easily define a Galois connection between 
the sets of functions assigning values to the abstract and the concrete states. 
Since there seems to be no easy way to adapt them or the ones from [15|16] to 
our construction, we show the soundness of our approach by a new proof that 
uses different techniques. 

We also propose an abstraction refinement technique that adapts the idea 
of delaying the application of widenings [5] to the probabilistic case. The tech- 
nique delays widenings at the nodes which are likely to have a larger impact in 
improving the bounds. We present experimental results on several examples. 

The paper is organized as follows. In the rest of the introduction we discuss 
related work and informally present the key ideas of our approach by means 
of examples. Section [2] contains preliminaries. Section [3] formally introduces the 
abstraction technique, and proves that games we are considering indeed give us 
upper resp. lower bound of the exact minimal and maximal reachability prob- 
abilities of reaching a set of states. Section [4] shows methods of refining our 
abstractions and discusses some experiments. 

Related work. Besides [12I13I15I21I22) , Monniaux has studied in [T7] how to ab- 
stract probability distributions over program states (instead of the states them- 
selves), but only considers upper bounds for probabilities, as already pointed out 
in [22]. In [18], Monniaux analyses different quantitative properties of Markov 
Decision processes, again using abstractions of probability distributions. In con- 
trast, our approach constructs an abstraction using "non-probabilistic" domains 
and widenings and then performs the computation of strategies and strategy val- 
ues, which might be used for a refinement of the abstraction. Finally, in [19 Han- 
kin, Di Pierro, and Wiklicky develop a framework for probabilistic Abstract In- 
terpretation which, loosely speaking, replaces abstract domains by linear spaces 
and Galois connections by special linear maps, and aims at computing expected 
values of random variables. In contrast, we stick to the standard framework, 
since in particular we wish to apply existing tools, and aim for upper and lower 
bounds of probabilities. 
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1.1 An example 

Consider the following program, written in pseudo code: 

int nrp = ; 
1: while (nrp < 100) 

2: if (rec_pack()) then nrp = nrp+1 else break 
3: if (nrp < 1) then (fail or goto 1) else exit 

where the choice between fail and goto 1 is decided by the environment. The 
goal of the program is to receive up to 100 packets through a network connec- 
tion. Whenever rec_pack() is executed, the connection can break down with 
probability 0.01, in which case rec_pack() returns false and no further packets 
can be received. If at least one packet is received (line 3) the program termi- 
nate^); otherwise, the program tries to repair the connection, which may fail or 
succeed, in which case the program is started again. The choice between success 
and failure is nondctcrministic. 

We formalize the pseudo code as a Nondeterministic Probabilistic Program, 
abbreviated NPFH (see Fig. Q]). A NPP is a collection of guarded commands. 
A guarded command consists of a name (e.g. Al), followed by a guard (e.g. 
(ctr = 1) & (nrp < 100) ) and a sequence of pairs of probabilities and update 
commands (e.g. 0.99: (nrp' = nrp+1)), separated by '+'. A reach-line at the 
end of the NPP describes the set of states for which we want to compute the 
reachability probability. We call them the final states. In our example, reaching 
fail corresponds to satisfying (ctr = 3) && (nrp < 1 ). A program execution 
starts with the initial configuration given by the variable declarations. The pro- 
gram chooses a guarded command whose guard is enabled (i.e., satisfied) by the 
current state of the program, and selects one of its update commands at ran- 
dom, according to the annotated probabilities. After performing the update, the 
process is repeated until the program reaches a final state. 

The probability of reaching fail depends on the behaviour of the environ- 
ment. The smallest (largest) probability clearly corresponds to the environment 
always choosing goto 1 (fail), and its value is (0.01). However, a brute force 
automatic technique will construct the standard semantics of the program, a 
Markov decision process (MDP) with over 400 states, part of which is shown in 
Fig. [TJ We informally introduce our abstraction technique, which does better 
and is able to infer tight intervals for both the smallest and the largest proba- 
bility. It is based on the parallel or menu-based predicate abstraction of [12121] , 
which we adapt to arbitrary abstract domains. 

1.2 Constructing a valid abstraction 

Given an abstract domain and a widening operator, we abstract the MDP of the 
program into four different stochastic 2-Player games sharing the same arena and 

2 It would be more realistic to set another bound, like 20 packets, but with one packet 
the probabilities are easy to compute. 

3 NPPs roughly correspond to a subset of the input language of the model checker 
PRISM pQ. 
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int nrp = , ctr = 1 ; 
Al: (ctr = 1) & (nrp < 100) 

-> 0.99: (nrp' = nrp+1) 
+ 0.01: (ctr' = 2) ; 
A2: (ctr = 1) & (nrp >= 100) 

-> 1: (ctr' = 3) ; 
A3: (ctr = 2) & (nrp < 1) 

-> 1: (ctr' = 1) ; 
A4: (ctr = 2) 

-> 1: (ctr' = 3) ; 
A5: (ctr = 3) & (nrp >= 1) 

-> 1: (ctr' = 3) ; 
reach: (ctr = 3) & (nrp < 1) 




Fig. 1. Example program and prefix of the corresponding Markov Decision Process. 
Actions are drawn as circles, program states (ctr, nrp) as rectangles. (3,0) is a final 
state. 
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Fig. 2. Abstraction of the program from Fig. \T\ 
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the same rules (i.e., the games differ only on the winning conditions). A round 
of the game starts at an abstract state n (we think of n as a set of concrete 
states) with Player 1 to move. Let n, be the set of concrete states of n that 
enable command Aj. Player 1 proposes an Ai such that rij ^ 0, modeled by a 
move from n to a node (n, Aj). If n contains some final state, then Player 1 can 
also propose to end the play (modeled by a move to (n, ©)). Then it is Player 2's 
turn. If Player 1 proposes Ai, then Player 2 can accept the proposal (modeled 
by a move to a node determined below), or reject it and end the play (modeled 
by a move to another distinguished node (g>), but only if Hi ^= n. If Player 2 
accepts Ai, she moves to some node (n,Ai,n') such that n' C m, i.e., Player 2 
can "pick" a subset n' of n, out of the subsets offered by the game arena (every 
concrete state in is contained in one such n'). 

The next node of the play is determined probabilistically: one of the updates 
of Ai is selected randomly according to the probabilities, and the play moves to 
the abstract state obtained by applying the update and (in certain situations) the 
widening operator to n' . If Player 1 proposes ® by choosing (n, ©), then Player 
2 can accept the proposal, (modeled by a move to ©) or, if not all concrete states 
of n are final, reject it (modeled by a move (n, ©) — > (n, ©)). 

Fig. [2] shows an arena for the program of Fig. [1] with the abstract domain 
and widening operator described in the following. Nodes owned by Player 1 are 
drawn as white rectangles, nodes owned by Player 2 as circles, and probabilistic 
nodes as grey rectangles. In the figure we label a node (n, Ai) belonging to Player 
2 with Ai and a probabilistic node (n, Ai, n') with n' (n resp. n and Ai can easily 
be reconstructed by inspecting the direct predecessors). Nodes of the form (n, ©) 
are labeled with '©?'. 

A (concrete) state of the example program is a pair (ctr, nrp), and an ab- 
stract state is a pair (ctr,[a,b\), where [a, 6] is an interval of values of nrp 
(i.e., ctr is not abstracted in the example). The widening operator V works as 
follows: if the abstract state (ctr, [a,b]) has an ancestor (ctr, [a',b'}) along the 
path between it and the initial state given by the construction, then we over- 
approximate (ctr, [a, b]) by (ctr, s), with s = [a' , b'] V [min(a, a'), max(6, b')]. For 
instance the node n = (1, [0,oo)) in Fig. [2] enables AI and A2, and so it has 
two successors. Since n contains concrete states that do not enable AI and con- 
crete states that do not enable A2, both of them have ® as successor. The node 
(n, AI, (1, [0, 99])) (whose label is abbreviated by (1, [0, 99]) in the figure) is prob- 
abilistic. Without widening, its successors would be (2, [0,99]) and (1, [1,100]) 
with probabilities 0.01 and 0.99. However, (1, [1, 100]) has (1, [0, oo)) as (direct) 
predecessor, which has the same cir-value. Therefore the widening overapprox- 
imatcs (1, [1,100]) to (1, [0, oo)V[0, oo)) = (1, [0, oo)), and hence we insert an 
edge from (n,Al, (1, [0,99])) (back) to (1, [0,oo)), labeled by 0.99. 

After building the arena, we compute lower and upper bounds for the minimal 
and maximal reachability probabilities as the values of four different games, 
defined as the winning probability of Player 1 for optimal play. The winning 
conditions of the games are as follows: 
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int c = 0, i = 0; 
if choice (0.5) then 



1 



int c = 0, i = 0; 
while (i <= 100) 



2 
3 
4 



while (i <= 100) 



2 
3 
4 



if choice (0.5) then i = (i+1) ; 



i = i+1; 



c = c-i+2; 
if (c >= i) then fail 
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c = c-i+2 
if (c >= i) then fail 



Fig. 3. Example programs 2 and 3. 



(a) Lower bound for the maximal probability: Player 1 wins if the play ends by 
reaching ©, otherwise Player 2 wins. 

(b) Upper bound for the maximal probability: Players 1 and 2 both win if the 
play ends by reaching ®, and both lose otherwise. 

(c) Lower bound for the minimal probability: Players 1 and 2 both lose if the 
play ends by reaching © or ®, and both win otherwise. 

(d) Upper bound for the minimal probability: Player 1 loses if the play ends by 
reaching © or (g), otherwise Player 2 loses. 

For the intuition behind these winning conditions, consider first game (a). Since 
Player 1 models the environment and wins by reaching ©, the environment's goal 
is to reach a final state. Imagine first that the abstraction is the trivial one, i.e., 
abstract and concrete states coincide. Then Player 2 never has a choice, and the 
optimal strategy for Player 1 determines a set S of action sequences whose total 
probability is equal to the maximal probability of reaching a final state. Imagine 
now that the abstraction is coarser. In the arena for the abstract game the 
sequences of S are still possible, but now Player 2 may be able to prevent them, 
for instance by moving to ® when an abstract state contains concrete states not 
enabling the next action in the sequence. Therefore, in the abstract game the 
probability that Player 1 wins can be at most equal to the maximal probability. 
In game (b) the team formed by the two players can exploit the spurious paths 
introduced by the abstraction to find a strategy leading to a better set of paths; 
in any case, the probability of S is a lower bound for the winning probability of 
the team. The intuition behind games (c) and (d) is similar. 

In our example, optimal strategies for game (b) are: for Player 1, always play 
the "rightmost" choice, except at (2, [0, 99]), where she should play A4; for Player 
2, play © if possible, otherwise anything but ®. The value of the game is 1. In 
game (a), the optimal strategy for Player 1 is the same, whereas Player 2 always 
plays ® (resp. stays in ©?) whenever possible. The value of the game is 0.01. 
We get [0.01, 1] as lower and upper bound for the maximal probability. For the 
minimal probability we get the trivial bounds [0, 1]. 

To get more precision, we can skip widenings at certain situations during 
the construction. If we e.g. apply widening only after the second unrolling of 
the loop, the resulting abstraction allows us to obtain the more precise bounds 
[0,0.01] and [0.01,0.01] for minimal and maximal reachability, respectively. 

The main theoretical result of our paper is the counterpart of the results 
of [15)12] : for arbitrary abstraction domains, the values of the four games de- 
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scribed above indeed yield upper and lower bounds of the maximal and minimal 
probability of reaching the goal nodes. 

In order to give a first impression of the advantages of abstraction domains 
beyond predicate abstraction in the probabilistic case, consider the (determin- 
istic) pseudo code on the left of Fig. [3j a variant of the program above. Here 
choice(p) models a call to a random number generator that returns 1 with 
probability p and with probability 1 — p. 

It is easy to see that c < 1 is a global invariant, and so the probability of 
failure is exactly 0.5. Hence a simple invariant like c < k for a k < 100, together 
with the postcondition i > 100 of the loop would be sufficient to negate the 
guard of the statement at line 5. However, when this program is analysed with 
PASS |12|13j . a leading tool on probabilistic abstraction refinement on the basis 
of predicate abstraction, the while loop is unrolled 100 times because the tool 
fails to "catch" the invariant, independently of the options chosen to refine the 
abstraction 0. 

On the other hand, an analysis of the program with the standard interval 
domain, the standard widening operator, and the standard technique of delaying 
widenings [5], easily 'catches" the invariant (see Section HIT]) . The same happens 
for the program on the right of the figure, which exhibits a more interesting 
probabilistic behaviour, especially a probabilistic choice within a loop: we obtain 
good upper and lower bounds for the probability of failure using the standard 
interval domain. Notice that examples exhibiting the opposite behaviour (pred- 
icate abstraction succeeds where interval analysis fails) are not difficult to find; 
our thesis is only that the game-based abstraction approach of |12|15j can be 
extended to arbitrary abstract domains, making it more flexible and efficient. 

2 Stochastic 2-Player Games 

This section introduces stochastic 2-Player games. For a more thorough intro- 
duction into the subject and proofs for the theorems see e.g. |2QI6l7j . 

Let S be a countable set. We denote by Dist(S') the set of all distributions 
S : S — > [0, 1] over S with 6(x) = for all but finitely many ieS. 

Definition 1. A stochastic 2-Player game Q (short 2-Player game) is a tuple 
((Vi,V2,V p ),E,S,8 ), where 

— Vi,V2,V p are distinct, countable sets of states. We set V = V\ U Va U V p ; 

— E<Z (Vi U V2) x V is the set of admissible player choices; 

— 5 : V p — >• Dist(V) is a probabilistic transition function; 

— sq G Vi is the start state. 

Instead of {q, r) G E we often write q —> r. A string w G V + is a finite run (short: 
run) of Q if (a) w — so, or (b) w — w's's for some run w's' G V*{V\ U V2) and 
s' — > s, or (c) w — w's's for some run w's' G V*V P such that S(s')(s) > 0. 

4 Actually, the input language of PASS does not explicitly include while loops, they 
have to be simulated. But this does not affect the analysis. 
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We denote the set of all runs of Q by Cyl(Q). A run w = x\ . . .Xk is accepting 
relative to F C V\ if Xk € F and Xi £ F for 1 < i < k. The set of accepting 
runs relative to F is denoted by C'yl(Q,F). 

A stochastic 2-Player game with V2 = is called a Markov Decision Process 
(MDP), and then we write Q = ((Vi, V p ), E, 5, s ). 

Fix for the rest of the section a 2-Player game Q = ((Vi, V2, V p ), E, S, So). The 
behaviours of Player 1 and 2 in Q are described with the help of strategies: 

Definition 2. A strategy for Player i G {1,2} in Q is a partial function (f> : 
Cyl{Q) — > Dist(V) satisfying the following two conditions: 

— 4>(w) is defined iff w = w'v £ V*Vi and v — > x for some x G V ; and 

— if 4>(w) is defined and (j)(w)(x) > then wx is a run. 

We denote the set of strategies for Player i by Si(Q). A strategy <j) is memoryless 
if 4>(w\) — 4>(w2) for any two runs w\,W2 ending in the same node ofVi, and 
non-randomized if for every run w such that (f)(w) is defined there is a node x 
such that 4>(w) (x) = 1 . Given strategies (f>\ , 02 for Players 1 and 2, the value 
val(w)g[ < j >1 ^ 2 \ of a run w under <pi,<p2 is defined as follows : 

— If w = s , then val{w)g[ <t>lt< j >2 \ = 1. 

— Ifw = w's G V*Vi for i G {1, 2} and <pi{w') is defined, then val(w)g[ ( j >1 _ l p 2 ] = 
val(w')g [4>u4)2 ] ■ 

— If w = w's's for some run w's' G V*V P then 
val(w)g [4>lM = val{w's')g [<Pu<p2] -5(s')( s )- 

— Otherwise val(w)g[ c f >1 ^ 2 ] = 0. 

We are interested in probabilistic reachability: 

Definition 3. The probability Reach(Q[<px, <p 2 ] 1 F) of reaching F C V\ in Q 
under strategies </>i and <p 2 of Players 1 and 2 is 

Reach(G[(/>i,(j>2],F) := ^ val ( w )gi<f>i,4>2]- 
wec y i(g,F) 

If the context is clear, we often omit the subscript of val (•). We write Cyl(£[0i, ^2]) 
(resp. Cyl(£/[^i, 2 ], F)) for the set of all finite runs r G Cyl(^) (resp. r G 
Cy\(Q,F)) with val (r)g[0 1 ^ 2 ] > 0. In a MDP M. we do not require to have a 
strategy for the second Player. Here we just write Reach(A / f[0i], F) for a given 
strategy G S\{M). 

Definition 4. Let Q = ((Pi, V2, V p ), E, 5, s ) be a 2-Player game, and F C V\. 
The extremal game values Reach(G, F) ++ , Reach{Q , F) + ~ , Reach(Q, F) h and 
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Reach(Q, F) are 

Reach{g,F) +A 

Reach(g, F) + ~ 

Reach(g,F)- A 

Reach(Q,F)— 



sup sup Reach(g[(pi,<f>2],F) 
0i£S , i(g)0 2 GS 2 (e) 

sup inf Reach(g[(pi,<j)2\,F) 

0i6Si(S) *2GS 2 (G) 

inf sup Reach(g[(pi,4>2],F) 
0ieSi(a) 2G s 2 (e) 

inf inf ReacblQ \4>i, 62], F) 
0ieSi(a)0 2 es 2 (e) 



is a MDP, we de/me Reach(g,F)+ := Reach(g,F)++ = Reach{g,F)+- and 
Reach{g,F)- := Reach{G , F)~ = Reach(g, F)~+ . 

The following well-known theorem will be crucial for the validity of our ab- 
stractions [BJ: 

Theorem 1. Let F c Vi- For each k G {++, H — , — h } there exist non- 
randomized and memoryless strategies (f)f G Si(Q),(f>2 G 82(g) such that 

Reach(g,F) K = Reach(g[(f>1, F). 

Extremal game values can be computed e.g. by variants of value iteration [7]. 



3 Abstractions of Probabilistic Programs 

We start by giving a formal definition of NPPs. 

Definition 5. LetV be a finite set of variables, where x G V has a range rng(x). 
A configuration (or state) of V is a map a: V — > U^ev rn 9i x ) such that a(x) G 
rng(x) for all x G V. The set of all configurations is denoted by £y. A transition 
is a map f G 2 Sv —> 2 Sv such that |/({c})| < 1 for all a G E\i (i.e., a transition 
maps a single configuration to the empty set or to a singleton again), and 

U /(M) = f(M) for all M C E v . 

a£AI 

A transition g is a guard if g({o~}) G {{c},0} for every configuration a. 
We say that a enables g if g({o~}) — {a}. A transition c is an assignment if 
\ c ({ a })\ — 1 f or a tt a £ 2Jv- The semantics of an assignment c is the map 
[cj : —> Sy given by [c](cr) := a' if c({o~}) = {o~'}. The set of transitions is 
denoted by Transy. 

Definition 6. Nondeterministic Probabilistic Programs. 

A nondeterministic probabilistic program (NPP) is a triple P — (V, <jq, C) where 
V is a finite set of program variables, cto G Sv is the initial configuration, and 
C is a finite set of guarded commands. A guarded command A has the form 
A = g — y pi : c% + . . . + p m ■ c m , where m > 1, g is a guard, pi, . . . ,p m 
are probabilities adding up to 1, and c\, . . . ,c m are assignments. We denote the 
guard of A by gA, the updates {(pi, Ci), . . . (p m , c m )} of A by up a, and the set 
{up A I A G C] by up c . 
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Definition 7. Semantics of NPPs and Reachability Problem. 
The MDP associated to a NPP P = (V,a ,C) is Mp = ((Vi, V p ), E, S, a ), 
where Vi = E v , V v = S v x C, E Cl^x (Vi U V p ), ^:(r v xC)^ Lfe^Vi), a«rf 
/or ewer?/ A 6 C, a, a' G £\> 



The reachability problem for P relative to a set F C U v of states such that 
(To ^ -F is the problem of computing Reach(A4 p , F) + and Reach(A4 p , F)~ . We 
call F the set of final states. 

We assume in the following that for every run wa G V*V\ in A4p either a G F or 
a enables the guard of at least one command (i.e., we do not 'get stuck' during the 
computation). This can e.g. be achieved by adding a suitable guarded command 
that simulates a self loop. 

3.1 Abstracting NPPs 

We abstract NPPs using the Abstract Interpretation framework (see [5]). As 
usual, an abstract domain is a complete lattice {D\ C, T, _L, U, (short D^), 
and we assume the existence of monotone abstraction and concretization maps 
a : 2 Sv — > D$ and 7 : — > 2 Sv forming a Galois connection between and 
2 Sv . A widen operator is a mapping V : D* x D* — y D» satisfying (i) aVo □ a 
and aV6 □ b for all a,b £ D", and (ii) for every strictly increasing sequence 
do C di C . . . in -D" the sequence (oi) zSN defined by bg = ao and o^+i = V a^+i 
is stationary. 

We abstract sets of configurations by elements of DK Following ideas from |15|12|13|21j . 
the abstraction of an NPP is a 2-player stochastic game. We formalize which 
games are valid abstractions of a given NPP (compare the definition to the com- 
ments in Section [TT2"j) : 

Definition 8. Let P — (V, o"o,C) be a NPP with a set F C U v of final states 
such that o"o ^ F '. A 2-player game Q = ((Vi, V%, V p ), E, <5, sq) wii/i finitely many 
nodes is a valid abstraction of P relative to F for Z?" i/ 

— Vi contains a subset of plus two distinguished states ®,®; 

— Vi is a set of pairs (s, A), where s € Vi \ {©> ®} either A = © or A is 
a command of C enabled by some state 0/7(3); 

— V^, is a se£ of fourtuples (s,A, s',d), where s,s' G Vi \ {©,(8)} smc/i £/ia£ 
s ] s', i is a command enabled by some state of"/(s'), and d is the mapping 
that assigns to every update (p, c) G up^ an abstract state s' G Vi wii/i 



(j — > (<t, A) ijff a - enables qa and S((a, A))(a') : 




P ■ 



(p,c)£up A : [c](o-)=CT' 



T (#,c)))2#')); 
s = a({a }); 



and the following conditions hold: 



1. For every s G Vi \ {©, <£)} and every A G C: 
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(a) Ifj(s) flF^tl then s -> (s, ©} -> ©. If moreover j(s) C F 7 i/ien (s, ©} 
is i/ie onZy successor of s; otherwise, also (s, ©) — > (s, ©) holds. 

(* If j(s) contains some final state, then Player 1 can propose ©. If 
all states of -f(s) are final, then Player 2 must accept, otherwise it can 
accept, or reject by staying in (s, ©). *) 

(b) Ifg A {l{s)) ^ then (s,A) e V 2 and s -> (s, A). 

(* If some state of -f(s) enables A then Player 1 can propose A. *) 

2. For every pair (s,A) G V2 and every A E C: 

(a) there exist nodes {(s, A, si, d\), . . . , (s, A, Sk, dk}} Q V p such that (s, A) — > 
(s,A,Si,di) for every i < k and g A (j(s)) C Uj = i7( s j)- 

(* If Player 2 accepts A, then she can pick any concrete state a e j(s) 
enabling A, and choose a successor (s, A, Si,d{) such that a £ 7(sj). *) 

(b) If^(s)r\F^9,then(s,A)^-@. 

(* If -f(s) contains some final state, then Player 2 can reject and move 
to ©. *) 

(c) If9A(l(s)) + 7(s), then (s,A) -> ®. 

(* If some state of -f(s) does not enable A, then Player 2 can reject A 
and move to <g>. *) 

3. For every (s,A,s',d) G V p and every abstract state s" e V\: 

6((s,A,s',d))(s"):= Yl P- 

(p,c)&upA - d((p,c))=s" 

4- The states <g> and © have no outgoing edges. 

We can now state the main theorem of the paper: the extremal game values 
of the games derived from valid abstractions provide upper and lower bounds 
on the maximal and minimal reachability probabilities. The complete proof is 
given in the appendix. 

Theorem 2. Let P be a NPP and let Q be a valid abstraction of P relative to 
F for the abstract domain DK Then 

Reach(M P ,F)~ E [Reach(Q,{©,®}) , Reach(Q, {©, ®})~ + ] and 
Reach(M P ,F) + g [Reach(G, {©})+-, Reach(G, {©})++}. 

Proof. (Sketch.) The result is an easy consequence of the following three asser- 
tions: 

(1) Given a strategy (j> 01 the (single) player in Mp, there exists a strategy 
4>i G Si(G) such that 

inf Reach(C/[<£i, V],{®,<8>}) < Reach(7W P [</>], F) and 

■4>es 2 (S) 

sup Reach(g[^!, V],{©}) > Rcach(7W P [<f>], F). 
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(2) Given a strategy <pi G Sx(Q) there exists a strategy G S^A^p) such that 

Reach(M P [(/)],F) < sup Reach(£[0i, ip], {<g>, ©}). 

(3) Given a strategy 0i G Si (Q) there exists a strategy <j> G ^(A^p) such that 

Reach(A4p[</>],F) > inf Reach(£[0i, {©})• 

To prove (1) (the other two assertions are similar), we use <f> to define a function 
D that distributes the probabilistic mass of a run R G Cy\(G) among all the runs 
r G Cyl(.M) (where Ai is a normalization of Aip). The strategies 4>i and 02 are 
then chosen so that they produce the same distribution, i.e., the mass of all the 
runs r that follow the strategies and correspond to an abstract run R following 
<j> is equal to the mass of R. □ 

Recall that in predicate abstraction the concretizations of two abstract states 
are disjoint sets of configurations (disjointness property). This allows to easily 
define a Galois connection between the sets of functions assigning values to the 
abstract and the concrete states: Given a concrete valuator /, its abstraction 
is the function that assigns to a set X the minimal resp. the maximal value 
assigned by / to the elements of X. Here we have to distribute the value of a 
concrete state into multiple abstract states (which is what we do in our proof). 



3.2 An Algorithm for Constructing Valid Abstractions 

Algorithm [1] builds a valid abstraction Q of a NPP P relative to a set F of final 
states for a given abstract domain DK It is inspired by the algorithms of |4I10) 
for constructing abstract reachability trees. It constructs the initial state sq = 
a({cro}) and generates transitions and successor states in a breadth-first fashion 
using a work list called work. The GENSUCCS procedure constructs the successors 
of a node guided by the rules from Def . [5J It uses abstract transformers and c" 
for the guards and commands of the NPP. Hereby a transformer g" : — > 2 Di 
abstracting a guard g has to satisfy that for all a G D% <?"(a) is finite and 
Ube ff »( a ) 7$) 3 g('j(aj). Allowing to return a set rather than just one element 
from can help increasing the accuracy of Q. Here we implicitly make use of 
abstract powerset domains. GENSUCCS assumes that it can be decided whether 
7 ( s ) n F = 0, y(s) 2 F, 9a{i{s)) ^ or #a(7(s)) ^ j(s) hold (lines 2 and 
4). The assumptions on F are reasonable, since in most cases the set F has a 
very simple shape, and could be replaced by conservative tests on the abstract. 
A conservative decision procedure suffices for the test gA(j(s)) ^ j{ s ), with 
the only requirement that if it returns 0, then 5,4(7(5)) = -f(s) has to hold. The 
same holds for the test <ju(7(s)) ^ 0. GENSUCCS closely follows the definition of 
a valid abstraction, as specified in Def. [H 

Lines 1 and 2 guarantee that condition (la) of Def. [8] holds, and, similarly, 
line 4 guarantees condition (lb). Similarly, lines 3 and 5 are needed to satisfy 



13 



conditions (2b) and (2c), respectively. The loop at line 6 generates the nodes of 
the form (s,A,Si,d) required by condition (2a) of our definition, and the loop 
at line 7 constructs the function d appearing in condition 3. 

As usual, termination of the algorithm requires to use widenings. This is the 
role of the EXTRAPOLATE procedure. During the construction, we use the function 
pred(-) to store for every node s G V% \{soj ©> ®} its predecessor in the spanning 
tree induced by the construction (we call it the spanning tree from now on) . For 
a node s' G V\ that was created as the result of chosing a guarded command A, 
the procedure finds the nearest predecessor s in the spanning tree with the same 
property, and uses s to perform a widen operation. Note that in the introductory 
example, another strategy was used: There we applied widenings only for states 
with matching control location. The strategy used in EXTRAPOLATE does not use 
additional information like control flow and thus can be used for arbitrary NPPs. 
We can now prove (see the appendix): 

Theorem 3. Algorithm]^ terminates, and its result Q is a valid abstraction. 



4 Refining Abstractions: Quantitative Widening Delay 

Algorithm [1] applies the widening operator whenever the current node has a 
predecessor in the spanning tree that was created by the application applying the 
same guarded command. This strategy usually leads to too many widenings and 
poor abstractions. A popular solution in non-probabilistic abstract interpretation 
is to delay widenings in an initial stage of the analysis [5] , in our case until the 
spanning tree reaches a given depth. We call this approach depth-based unrolling. 
Note that if Aip is finite and the application of widenings is the only source of 
imprecision, this simple refinement method is complete. 

A shortcoming of this approach is that it is insensitive to the probabilistic 
information. We propose to combine it with another heuristic. Given a valid 
abstraction Q, our procedure yields two pairs ((f>i i^t) resp. (4>i ,4>2) °f rnem- 
orylcss and non-probabilistic strategies that satisfy Reach(C/[0f , </>j], {©}) = 
Reach(C?, {©}) H resp. Reach(£[(/>+,^+],{©}) = Reach(5, {©})++. Given a node 
s for Player 1, let P s + and P~ denote the probability of reaching © (resp. © or 
® if we are interested in minimal probabilities) starting at s and obeying the 
strategies (</>i~,<^2~) resp. (<^,</>^~) in Q. In order to refine Q we can choose any 
node s e Vi fl D' such that P+ - P~ > (i.e., a node whose probability has 
not been computed exactly yet), such that at least one of the direct successors 
of s in the spanning tree has been constructed using a widening. We call these 
nodes the candidates (for delaying widening). The question is which candidates 
to select. We propose to use the following simple heuristic: 

Sort the candidates s according to the product w s ■ (P s + — P~ ) , where w s 
denotes the product of the probabilities on the path of the spanning tree 
of Q leading from so to s. Choose the n candidates with largest product, 
for a given n. 
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int a=0, ctr=0; 
Al: (ctr=0) 



int x=0, y=0, c=0; 
Al: (c=0)&(x<=1000) 



-> 0.5: (a'=l)fc(ctr'=l) 



-> 0.25: (x'=3*x+2)&(y'=y-x) 



+0.5: (a'=0)&(ctr'=l) ; 



+0.75: (x'=3*x)&(y'=30) ; 



A2: (ctr=l)&(a>=-400)&(a<= 400) 



A2: (c=0)&(x>1000) -> l:(c'=l); 
A3: (c=l)&(x>=3) -> l:(x'=x-3); 
reach: (c=l)&(x=2)&(y>=30) 



-> 0.5: (a'=a+5) 



+0.5: (a'=a-5) ; 



A3: (ctr=l) -> l:(ctr'=2); 
reach: (a=l)&(ctr=2) 



Fig. 4. Two guarded-command programs. 



We call this heuristic the mass heuristic. The mixed heuristic delays widenings 
for nodes with depth less than a threshold i, and for n nodes of depth larger than 
or equal to i with maximal product. In the next section we illustrate depth-based 
unrolling, the mass heuristic, and the mixed heuristic on some examples. 

4.1 Experiments 

We have implemented a prototype of our approach on top of the Parma Polyhe- 
dra Library [3], which provides several numerical domains [2]. We present some 
experiments showing how simple domains like intervals can outperform predi- 
cate abstraction. Notice that examples exhibiting the opposite behaviour are also 
easy to find: our experiments are not an argument against predicate abstraction, 
but an argument for abstraction approaches not limited to it. 

If the computed lower and upper bounds differ by more than 0.01, we select 
refinement candidates using the different heuristics presented before and rebuild 
the abstraction. We used a Linux machine with 4GB RAM. 
Two small programs. Consider the NPPs of Fig. |H We compute bounds with 
different domains: intervals, octagons, integer grids, and the product of integer 
grids and intervals [5]. For the refinement we use the mass (M) depth (D) and 
mixed (Mix) heuristics. For M and Mix we choose 15 refinement candidates at 
each iteration. The results are shown in Table [T] For the left program the inte- 
ger grid domain (and the product) compute precise bounds after one iteration. 
After 10 minutes, the PASS tool [T2J only provides the bounds [0.5, 0.7] for the 
optimal reachability probability. For the right program only the product of grids 
and intervals is able to "see" that x = (mod 3) or y < 30 holds, and yields 
precise bounds after 3 refinement steps. After 10 minutes PASS only provides 
the bounds [0,0.75]. The example illustrates how pure depth-based unrolling, 
ignoring probabilistic information, leads to poor results: the mass and mixed 
heuristics perform better. PASS may perform better after integrating appropri- 
ate theories, but the example shows that combining domains is powerful and 
easily realizable by using Abstract Interpretation tools. 

Programs of Fig. [3j For these PASS does not terminate after 10 minutes, 
while with the interval domain our approach computes the exact value after at 
most 5 iterations and less than 10 seconds. Most of the predicates added by 
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PASS during the refinement for program 2 have the form c < a ■ i + j3 with 
a > 0,(3 < 0: PASS'S interpolation engines seem to take the wrong guesses 
during the generation of new predicates. This effect remains also if we change 
the refinement strategies of PASS. PASS offers the option of manually adding 
predicates. Interestingly it suffices to add a predicate as simple as e.g. i > 3 to 
help the tool deriving the solution after 3 refinements for program 2. 
Zeroconf. This is a simple probabilistic model of the Zeroconf protocol, adapted 
from |l|15j . where it was analyzed using PRISM and predicate abstraction. It is 
parameterized by K , the maximal number of probes sent by the protocol. We 
check it for K — 4,6,8 and two different properties. Zeroconf is a very good 
example for predicate abstraction, and so it is not surprising that PASS beats 
the interval domain (see Table [2|. The example shows how the mass heuristic by 
itself may not provide good results either, with depth-unrolling and the mixed 
heuristics performing substantially better. 

5 Conclusions 

We have shown that the approach of [15122] for abstraction of probabilistic sys- 
tems can be extended to arbitrary domains, allowing probabilistic checkers to 
profit from well developed libraries for abstract domains like intervals, octagons, 
and polyhedra [3|14j . 

For this we have extended the construction of abstract reachability trees 
presented in [10] to the probabilistic case. The extension no longer yields a tree, 
but a stochastic 2-Player game that overapproximates the MDP semantics of 
the program. The correctness proof requires to use a novel technique. 

The new approach allows to refine abstractions using standard techniques like 
delaying widenings. We have also presented a technique that selectively delays 
widenings using a heuristics based on quantitative properties of the abstractions. 
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Appendix 

The key of the proof of Theorem [5] is the following lemma: 



Lemma 1. Let P = (V, o~o,C) be a NPP and Q a valid abstraction of P relative 
to F for the abstract domain . Then 
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(1) Given a strategy <f> of the (single) player in Mp, there exists a strategy 
4>i £ Si(G) such that 

inf #eac/j(<?[0i, V>],{®,<8>}) < Reach(M P [(p} , F) < sup Reach(Q[(j)i, tp], {©}). 
V>eS 2 (S) ~ ipeS 2 (g) 

(2) Given a strategy <\>\ G <Si(<?) £/iere exists a strategy <fi S <Si(.Mp) smc/i £/iai 

i?eac/i(Xp [0] , F) < sup Reach(Q[(/)i, ip], {(8), ©}). 
V>es 2 (s) 

(^J Given a strategy <p\ G Si (£) #iere exists a strategy <ft G 5i(A / (p) smc/i t/iat 

Eeac/i(A1p[^],F) > inf Reach(Q\4>^ {©})■ 

■tf>&S 2 (G) 

Proof. In the proof we will use some additional notation to keep the presentation 
short. For a guard g and a configuration a over V we write a \= g iff g enables 
a. We will also write Cyl(.M[0], ^F) as an abbreviation for the set Cy\(M [</>]) \ 
Cyl(MM,F). 

Let g = ((Vj^V^p),^, <S, s ), and M P = {{V u V p ), E, S, a Q ). We set V = 
Vi U V 2 U Vp and V = V\ UV P . In the following, we denote by R,R,R,... elements 
in Cy\(G), by r, f , f, . . . elements in Cyl(.M p), by s, s, s, . . . elements in I7 and 
by a, a, a,... elements in V \. 

We first note that runs R G Cy\(G[<j)i, 2 ]) that end in a node in V\ always can 
be composed in R = R1R2, with i?i G Vi^VpVi)* and R2 the empty word or 
R2 S V^j®,®}. Analogously, every run r G Cyl(A4p) ending in V\ is contained 
in Vi(V p Vi)* . We will exploit this structure in the proof. 

Proof of (1). We modify Mp as follows: We add a new node 07 to V\ and 
edges (q, 07) to E for every q E F, and remove all outgoing edges of nodes in F. 
We denote this modified MDP by M. Note that we have Reach(A / J [(f)], {07}) = 
Reach(A / (p [0] , F) for every strategy <p S Si(.M), because the only choice of 
for a history ending in cr G F is 07 (and every strategy <p £ S'i(Alp) corresponds 
uniquely to a strategy <fi' G Si(Ai), where 4>' chooses 07 in every node a G F, 
which is the only option there for (f>'). 

We show that there exist strategies <p\ G S\(Q) and <p 2 G 82(G) such that 

Reach(.M [</>], {<!/}) = Rcach(0[<£i, fa], {©}), 

which implies Reach(£[0i, 2 ], {©}) > Reach( A4p [</>] , {07}). Since 02 in our 
proof never chooses <g> we also get Reach(£/[</>i, </> 2 ], {©, <S>}) < Reach(A / (p [0] , {07}). 

The crucial point of the proof is to distribute the probabilistic mass of a run 
R G Cy\(G) (ending in a node of the first player) among all the runs r G Gy\(M) 
in a suitable way, depending on the strategy </>. This distribution is then used to 
define the strategies 4>\ and </> 2 . We formalize the distribution as a function 

D : (Cyl(S) n V*V\) x (Cyl(M) n V*Vi) -»• [0, 1]. 
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where, loosely speaking, D(R, r) is the fraction of the probabilistic mass of R 
that is assigned to r. In order to define D we need an auxiliary function /?. Due 
to the requirements of a valid abstraction, we know that for every (s, A) e V2 
and every concrete configuration a 6 j(gA(s)) there exists at least one node 
(s,A, s',d) <E V p with (s, A) — > (s,A,s',d) and <r G j(s'). But there might be 
more than one node in V p satisfying these conditions. We fix for every (s, A) and 
every a G 7(<7a(s)) an arbitrary successor node {s,A,s',d) with a G "f(s') and 
set P(a,s,A) = (s,A,s',d). 

Now we finally proceed to define D inductively: 

(i) If R = so and r = cr then D(R, r) := 1. 

(ii) If R = R(s,A)(s,A,s',d)s, r = f(a,A)a, AeC and f}(<j,s,A) = (s,A,s',d) 
then 

D(R,r):= ]T <f>(r)((a, A)) ■ D(R,r) ■ p. 

d((p,c»=sA[c](cr)=<7 

(iii) If R — Rs{s, ©)© and r = faa s then D(R, r) := D(Rs, fa). 

(iv) Otherwise D(R, r) := 0. 

Note that, due to the properties of a valid abstraction and our definition of 
D, if D(Rs,ra) > 0, then a G -f(s) for every ct G £y. Also note that for 
every f G Cyl(.M) We now proceed to define the strategies </>i,</>2- We use the 
abbreviations 

D R ■■= £r€Cyl(A<M) D ( R ' r ) and D r ~ E R eCyl(d[^ 2 ]) D ( R > r ) 

Given a run R G Cyl(<?) fl V*V\, if Dr = wc define <pi{R) arbitrarily, i.e., we 
let 4>\{R) be an arbitrary successor of the last node of R. If Dr ^ 0, we define 
(j)i(R) as follows: 

MR)((s,A)) := ±- ■ ]T D(R,r) ■ d>(r)((a, A)) 

R reCy\(M{<t>],^F) 

MR)((s,®)):=^-- ]T D ^ r ) 

R reCy\(M{4>\,F) 

Given a run R = R(s, A) G Cyl(£) n V*V 2 and A G C, if D R = wc define 
4>2(R) arbitrarily. If D^, ^ 0, we define 4>2{R) as follows: 

2 D(R,r) ■ (j)(r)((a,A)) 

reCy\(M[4>]^F): 

MR(s,A))((s,A,s',d)) := 

-Dfl-0l(i?)((s,A)) 

02 ©))(©) := 1 

It is easy to see that the functions (f>\, 4*2 so defined are indeed strategies: Recall 
that a NPP cannot reach a configuration a £ F where no guarded command is 
enabled. Hence cf){r) is always defined when used in the definition. 
We list several properties of the function D: 
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(a) For every r G Cyl(M[<f>],{a f }): if D(R,r) > 0, then R G Cy\(Q[<h.,<fo],{@})- 
(Follows easily from the definitions.) 

(b) For every R G Cyl(0[&, <fc], {<§>}): if £>(iJ,r) > 0, then r G Cyl(jH[<fl, {<t,}) 
(Follows easily from the definitions.) 

(c) For every R G Cyl((/[0i, <£ 2 ]) ending in Pi U {<§), ®}: = val (R). 
(Proof delayed, see below. Note that due to our choice of <f>i, 4>2, <8> can never 
be reached.) 

(d) For every r G Cyl(<?[^i, ^2]) ending in V\ U {07}: D r = val (r). 
(Proof delayed, see below.) 

Using (a)-(d) we can now conclude the proof: 
Reach(e[0i,0 2 ],{®}) = val(.R) (Def. of Reach()) 

-ReCyl(S[0i,0 2 ], {<§>}) 

E E (Prop, (c)) 

KeCyl(S[0i,0 2 ], {<§>}) r£Cyl(A4[0]) 

2 Yl D ^ r ) (Props, (a)-(b)) 

reCyl(M[4>],{vf}) fleCyl(g[</>i ,<fe]) 

53 val (r) =Reach(.M [(/>]) (Prop. (d)). 

reCylCMM,^/}) 

Proof of property (c). By induction on the length of i?. 

- If i? = s then EreC y i(MM) jD ( s o,0 = D(s a ,a a ) = 1 = val (R)g [< p u4>2 ]. 

- If R = R(s, A) (s, A, s',d)s G Cyl(0[<£i, <fe]) then 

val 

= val(fl)-^i(fl)((s,i4»-^ 2 (fl(s,i4»-<5((s,i4,s / ,d))(5) (Def. val(-)) 

= D R ■ 0i A)) • 2 (i?( S , A, s', d)) • <5« S , A, s', d))(g) (Ind. Hyp.) 

= ( E D(^,f)-0(f)((tr,i4))j ■<5((*,A,«',d))(5) (Def. fc,^) 

\f(CT,A)eCyl(A4M,-.F): / 
/3(o-,s,A)=(s,A,s',d> 

= E ( E p f)-0(f)«a,^») (Def. 5) 

<p,c)eup A : \f<<7,A>eCyl(A4M,-.F): / 
d«p,c))=s P(a,s,A) = (s,A,s',d) 

= E ( E p-D(R,r)-<j>(f)({<?,A})) (**) 

d((p,c))=s /3(<r,s,A)=(s,A,s',d> 
A[c](<r)=£r ^_ 

E E p-Wr)-0(f)((a,A)) 

f(cr,A)<xeCyl(.A/l[0],^-F) {p,c)£up A : 

d«p,c»=SA[c]( CT )=cr 
A/3((T.s,A) = (s.A.s'.d) 

= £> fl . (Def. D). 
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(**) For every run f(a, A) and every (p, c) e C there exists exactly one 
extension r(<7, A) [c](cr), and so the value of the sum does not change by 
modifying the sum quantifier in this way. 

- If R = Rs(s, ©)® 6 Cyl(£[0i,0 2 ]) then 

val (i?) = val (J2S) • 0i(jta)((S, ©)) • 02(ftS(S, ©})(©) 

= D fe -0i(^S)«5,@))-^2(fl5<5,@»(@) (Ind. hyp.) 

= D Rs • 0i (£*)((*, ©)) {MRs(s, ©))(©) = 1 by dcf. of 0a) 

E D(Rs,r) (Dcf. ofDij) 

reCyl(A4M,F) 

E D(Rs(s,@)@,r<Tf) (Dcf. of £>, part (hi)) 

reCyl(A4M,F) 

= Ar. (Dcf. of £>, Props, (a)-(b)) 

Proof of property (d). By induction on the length of r. 

- If r = ao then we proceed as in the case of i?. 

- Ur = r(a,A)aeCyl(M[<f>]) then 

val (r) 

= val(f) -0(r)((a,A)) ■ E P 

<P,c)G«pa:[c](ct)=ct 

= ( E r>(^,f)J -^({a.A))- ^ P (Ind. Hyp.) 

\-Rs€Cyl(a[4>i,cfe]) ' <P,c)G«p^:[c](CT)=cr 

E E £(^,r~).«Xr)((<^)).p 

HseCyl(a[0i,<^ 2 ]) <P,c>Gm PA : 
[c](<t)=<t 

E E D(R8,f)-4>(f)((a,A)).p (**) 

Rs{s,A)(s.A.s' .d) {p,c)eup A -- 
eCy\(Gl4>i,4>2]): [c](cr)=(T 
/9(er,s,A) = (s,A,s',d) 

E E ^r).0(r)((a,^)).p (***) 

Ks(s,A)(s,A,s',d)I (p,c)eup A : 
eCyl(0[0i : 02]): [c](cr)=<7 
/9(er,s,A) = <a,A,s',d) Ad((p,c)) = s 

E E D{Rs,r) ■<(>{?){{*, A)) -p 

Rs{s,A)(s,A,s' ,d)s (p,c)£up A : 
eCyl(S[^i,^ 2 ]) d({p,c))=s/\{c\(a)=<j 
A/3(a,s,A) = (s,A,s',d) 

= D r . 

(**) If D(Rs,r) > 0, then 5 € ■y(s) holds. Using the definition of a valid ab- 
straction we conclude that s — > (s, A) if u — \ (a, A) (especially if 0(r)((<7, A)) > 
0). Also there exists a unique node (s, A, s', d) with (s, A) — »■ (s, A, s', d) and 
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(s, A, s',d) = /3(<t, A, s). Therefore we can replace the sum quantifier without 
changing the summands here. 

(***) For every t = (p, c) G up a , there exists a unique s € V\ such that d(t) = 
$ and {s,A,s',d) —> s. So, for a fixed Rs(s, A)(s, A, s', d) with /3(a,s,A) = 
(s,A, s',d), 

J2 D{Rs,f)-<P{f){{5,A))-p= J2 E D(Rs,r)-4>(r)({a,A})- 

[c](or)=<7 Rs{s.A){s,A,s' .d)s [c](ct)=<t 

eCyl(a[<Ai,^ 2 ]) Ad«p,c»=s 

— If r = f<707 then 

val (r) = val (fcr) • <f>(fa)(af) 

= val (m) (**) 

= £>f£r (Ind. hyp.) 

= D r (Def. ofD). 

(**) By definition of M. 1 the only successor of a is 07. 



Proof of (2). Recall that at the beginning of the proof of (1) we have extended 
M.p by adding a new node 07 to V\ and by adding edges ((?, 07) to £7 for every 
q G F. We now extend .Mp further by also adding edges (q,<Tt) to E for every 
gS7i\f. Now every node in Vi \ {07} has at as a successor. We still call this 
modified MDP M. 

Let <j>\ G Si (Q) be a strategy for Player 1. We show that there exist strategies 
02 G S 2 (G) and <p G Si(.M) such that 

Reach(.M [</>], {07}) = Reach(<7[0i, 2 ], {©, <»})• 

To show that this equality proves (2), observe that for every <f> G S\(Ai) there 
exists a strategy </>' G Si(Mp) such that Rea,ch(Mp [<f>'], F) < Reach(A / ([0], {07}) 
(simply distribute the probability assigned to 07 to other successors arbitrarily 
if necessary). So for every </> G Si(A4) that satisfies the equality we have 

Reach(7W[0'],F) < Reach(^[0i,0 2 ],{®,®}) < sup Reach(a[</>i, V>], {®, ©}). 

</>eS 2 (S) 

We now observe that it suffices to prove the results for strategies 0i that are 
memoryless and non-probabilistic: this follows easily from Theorem [TJ which 
shows that the infimum over all <f>i of sup^ eg2 /g) Reach(C/[0i, ip], {®, ©}) is 
achieved for such a strategy <pi . Hence as an abbreviation we can write (j)i(s) = A 
for c/>i(Rs)((s,A}) = 1 for all s G Vi, Rs G Cyl^^i, <fc]) and a G C U {0, ©}. 

For the proof we again define a suitable distribution function D, this time as 
follows: 

— If R = (T and r = a then D(R, r) := 1. 



22 



- If R = R(s,A)(s,A,s',d)s and r = r(a,A)a, with (s,A) -ft <g> and (s,A) ft 
© and (3(a, s, A) — (s, A, s', d) then 

D(R,r) := D(R,f) • ]T p. 

d«P,c»= S A[c] 

- If _R = #(s, A)® with (s, A) ->• ® (with ,4 e C U {©}) and r = mo/ then 

D(R,r) := D(R,ra). 

- If = #(s,A)(g) with (s, A) ->■ ® and (s,A) © (with ieCU {©}) and 
r = facr/ then 

D(R,r) := D(R,ra). 

- Otherwise D(R, r) := 0. 

We now proceed to define 4>. Let final((s,A)) denote that (s, A) — > © or 
(s,A) ->■ (8). Let m G Cyl(jW). 

- If D r(X = then we define <f>(r) arbitrarily. 

- If D ra > then for every AeC 

fara)((a,A)) := J- £ £>(i? S ,m) 

rCT -RseCyl(e[0i,0 2 ]): 
#i(s)=.AA-i/inaJ((s,A)) 

and 

<t>{r&){(Tj):=^- J2 D(Rs,ra) 

Ta i?sGCyl(e[0i,0 2 ]): 
<£i(s)=AA/iraa/((s,A)) 

Note here that this is a valid strategy: If <r ft (a, A), then we know (due 
to the definition of a valid abstraction and again the fact that a <G j(s) if 
D(Rs,ra) > 0) that (s,a) — > ®, hence fara)((a, A)) = 0. Also it is easy to 
see that far a) (a f) + J^Aec < / , ( rfJ )(( cr ^)) = 1- 

Finally, we define fa 6 82(G)- As in part (1), 2 directs the probabilistic 
mass to the matching nodes in V p relative to (3. But in this case we choose © 
resp. ® as often as possible: For R = Rs(s,a) E Cy\(Q[fa, fa]) (i.e., fa(s) = A), 

- If (s,A) ->• © then fa(R)(@) = 1. 

- If (s, A) © and (s, A) -> ® then fa(R)(<2>) = 1. 

- If (s,^4) ©, (s, A) ft ®, and Z)^ = then we define fa(R) arbitrarily. 

- If (s,A) -ft ©, (s,A) -ft <g>, and £> fl > then for every (s,A,s',d) with 

-> (s,A,s',d) 

fa(R)((s,A,s',d)) = ^-- D(Rs,ra). 

R raeCyl(M): 

P(a,s,A)={s,A,s' ,d) 
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Now, as in the proof of (1), we show that with this choice for D, <j>, and (j) 2 
the properties (a)-(d) hold. The rest of the proof is then exactly as in the proof 
of (1). Proving properties (a) and (b) is again easy. 

Proof of property (c). By induction on the length of R. 



- If R = s then val (R) = 1 = D(R, cr ) = Dr. 

- For R = Rs(s 7 A)(s, A, s', d)s then the proof proceeds as for (1). 

- For R = Rs(s,A)@, (f>i(s) = A and (s,A) -> ©: 



val (R) = val (Rs) ■ <fo(Rs(s, A))(@) 
= val (R) 



val (R) = val (Rs) ■ <fc(Rs(s, A))(®) 
= val (Rs) 



<j) 2 (R§(§,A))(©) = 1 by Dcf. of 2 
(Ind. hyp.) 

(Def. of D R and Def. of D, part (iii)) 
= D R . 



<j> 2 (Rs(s,A))(®) = 1 by Dcf. of 4> 2 
(Ind. hyp.) 

(Def. of D R and Def. of D, part (iv)) 



If (s, A) ©, then val (R) = val (Rs) ■ (f> 2 (Rs(s, A))(@) = 
- For R = Rs(s,A)®, (f>i(s) = A, (s,A) -ft © and (s,A) ->■ ®: 



If (s, A) ->■ © or (.?, A) ft ® holds, then val (i?) = val (Rs)-4> 2 (Rs(s, A))(<g>) = 

= Dr. 



Proof of property (d). By induction on the length of r. 



— If r = (To then val (r) = 1 = D(so, r) = D r . 
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If r = fa (a, A) a then 
val (r) 

= val (fa) ■ <t>(fa)((a, A))- 53 P ( Dcf - val (0) 

(p,c)£up a : 
[c](cr)=<7 

= D P9 ~- Yl D(Rs,fa)- Yl P (Ind. hyp., Def. 0) 

Ufa ~ , . 

flseCyl(5[0i,^ 2 ]): (p,c)e«PA: 
^i(s)=AA-./inoi((s,A» [c](er)=<7 

^ D(Rs,fa)- Y P 

Rs£Cyl(gl<pi.<f> 2 ]): {p,c)£up a : 
4 >1 (s)=A^final((s,A)) [c](?)=<7 

53 D(^,m)- P (**) 



fls(«, J 4,s',d>eCyl(e[0i,02]): {p,c)£up, 
^i(s)=AA-./moJ«s,A» [c](" 
A/3(cr,s,A) = <s,A,s',d> 



O" — O" 



X] E P-D(Rs,fa) 



Rs{s,A,s',d)€Gyl(g[<t>i,<t>2]): {P,c)eu Pa : 
4 >1 (s)=AA,^final({s,A)) [c](er)=<r 
A/3(5-,s,A) = (s,A,s',d> 

= 53 53 P ' ^) ( see explanation in part (1)) 

fls(s,A,s',d)S€Cyl(a[4>i,0 2 ]): <P,c)s«pa: 
^i(s)=AA-./inoK(s,A» [c](er)=(7 
A0(cr,s,A)=(s,A,s',d) Ad«p,c))=s 

= L» r (Dcf. of£>). 

(**) As in part (1), we use that there exists a unique (s, A, s',d) with 
0(5, a, A) = (s,A,s',d). 
— If r = faaf then 

val (r) 

= val (fcr) • 4>(fa)(af) (Def. val (•)) 

= D ™-^Z' E Z3 ( i? ' r ) (Ind. hyp., Def. 0) 

rCT fleCyl(e[0i,0 2 ]): 

01 (s)-AA/mal((s,a)) 

53 D(ii,r) 

•ReCyl(S[0i,0 2 ]): 
4>i (s)=A/\final((s,a)) 

= D r (Def. of D r and D) 

Proof of (3). This proof is similar to the one of part (2), but now Player 2 
always chooses <g> if she can. Therefore we add an additional node cr® to M 
which is no goal state, and connect every node in V\ with a®. These edges now 
simulate the choice of <g> as successor node in G[<f>i,<t>2]- The rest of the proof is 
a simple variation of the one of part (2). 
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We can now proceed to proving the theorem: 

Theorem 2. Let P be a NPP and let Q be a valid abstraction of P relative to 
F for the abstract domain . Then 

Reach{M P ,F)~ G [Reach{G, {©, <g>}) , Reach{Q, {©, ®})~ + ] and 
Reach{M P ,F)+ G [Reach(g, {®}) + ~, Reach(g, {©})++]. 

Proof. Let ^ G S^A-f ) be a strategy of the (single) player in M. P . By Lemma[IJl) 
there exists a strategy <j>\ G S\{Q) satisfying 

inf Reach(g [01 >],{©,»}) < Reach(X P [S, F). 

From this we conclude that for all 4> G Si(Aip) 

Reach(£,{©,<g>})-- < inf Reach(£[<^, ip], {©, ®}) < Reach(A4 P [</>], F) 

and hence 

Reach(£,{®,<g>})-- < inf Reach( A4 P [0] , F) = Reach(A4 P , F)". 

4>es 1 (M P ) 

The inequality Reach(A^p, F) + < Reach((J, {©}) ++ can be proved in the same 
way by using the left inequality of Lemma [UJl). The remaining inequations 
Reach(X P ,F)- < Reach(£, {©, ®}) _+ and Reach(A4 P , F)+ > Reach(£, {©, ®}) 
are proved similarly using Lemma [Tf 2) and Lemma HJ3), respectively. 

Proof of Theorem [3] 

Theorem 3. Algorithm^ terminates, and its result Q is a valid abstraction. 

Proof. Assume for the sake of contradiction that Algorithm Q] does not termi- 
nate. Since every node in Q has only finitely many successors, the spanning tree 
of Q contains an infinite branch so — % s\ — i> S2 . . . by Konig's lemma, and at 
least one action a G C appears infinitely often in the branch, since C is finite. 
Let qo, to, qx, t\ . . . be the sequence of all nodes in the branch such that qi — > i; 
for all I G N. Then, by the definition of EXTRAPOLATE, there exists a sequence 
t>o, vx, . . . of elements in Z?" such that U + x = t{S7{vi U t{) □ t\ for all I > 0, and 
so to E tx E • • ■ holds. Define ao := to and az+i := U U for Z > 0. A simple 
induction shows that this sequence is monotonically increasing. Since tx = ax 
and for / > it holds that ti+x — t(S7{vi U tf) — t/Va;+i, we conclude from the 
definition of a widening operator from Section [3. II that there is a number k such 
that tk = tk+x, a contradiction to the assumption that the branch is infinite 
(since then we would have a cycle). 

We already pointed out that every newly constructed node in Q satisfies the 
conditions from Def. [8] and EXTRAPOLATE(a) □ a holds for all a£D', we can 
conclude that Q is a valid abstraction. 
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Algorithm 1: Computing Q. 

Input: NPP P = (V, (To,C), abstract domain D*, set of final states F C Ey, 
widening V. 

Output: 2-Player game Q = ((Vi,V 2 ,V p ), E, 8, s ). 

s = a({(j }); pred(s ) <- nil 
Vi <- {s ,<S>, ©}; V 2 ^ 0; V p <- 0; work <- {a } 
while worfc 7^ do 
|_ Remove s from the head of work; GENSUCCS(s) 



Procedure GENSUCCS(s € Vi) 

fopt 4— false 

if 7(a) n F j= then 

£<-£U{(a,(a,®»,«a, ©>,©)} 
if 7(a) g F then { £ <- E U {((a, ©>, (a, ©))}; fopt 
else return 



true } 



forall the A G C do 

if sa(tOO) 7^0 then 

C 2 <- K 2 U {{s, A)}; E+-EU {(a, (a, A))} 
if <m(tO)) / 7(s) then E E U {((s, A),®)} 
if fopt then E 4- E U {((a, A), ©)} 

forall the a' G 5a ( s ) do 

Create a fresh array d : upc — > Vi 
forall the (p, c) G wpA do 

v 4- EXTRAPOLATE(c*(s), s, A); d((p,c)) 4- v 
if v Vi then { 

Vi ^— Vi U {v}; prcd(u) = (s, A); add v to work } 
V p <- V p U {(a, A, a', d»; B «- E U {((a, A), (a, A, a', d))} 

Procedure EXTRAPOLATE^ G D", s G Vi \ {©, G C) 

<a',A') <- pred(s) 
while pred(s') 7^ niZ do 

if A' = A then return sV(s U v) 
else { buffer <- a'; (a', A') <- pred(a'); s «- buffer } 



return tj 
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Program 


Value 


Interval 


Octagon 


Grid 


Product 






M 


D 


Mix 


M 


D 


Mix 


M 


D 


Mix 


M 


D 


Mix 




Iters: 


23 


81 


24 


28 


81 


28 


1 


1 


1 


1 


1 


1 


Left 


Time: 


25 


66.1 


27.6 


26.6 


63.2 


26.9 


0.39 


0.39 


0.39 


0.6 


0.6 


0.6 


Size: 


793 


667 


769 


681 


691 


681 


17 


17 


17 


61 


61 


61 




Iters: 




















3 


7 


3 


Right 


Time: 




















8.3 


20.3 


8.2 


Size: 




















495 


756 


495 



Table 1. Experimental results for the programs in Fig. [4] Iters is the number of 
iterations needed. Time is given in seconds. '-' means the analysis did not return a 
precise enough bound after 10 minutes. Size denotes the maximal number of nodes 
belonging to Player 1 that occured in one of the constructed games. 



Zeroconf protocol 


K — 4 


K = 6 


K = 8 


K 


= 4 


K = 6 


K = 8 


(Interval domain) 


PI 


PI 


PI 




P2 


P2 


P2 


Time (Mass heuristic): 


6.2 


16.8 


32.2 




5.8 


18.5 


50.6 


Time (Depth heuristic): 


2.6 


6.0 


6.6 




2.6 


6.7 


8.1 


Time (Mix): 


2.6 


6.3 


6.8 




2.6 


6.9 


8.4 


Time PASS: 


0.6 


0.8 


1.1 




0.7 


0.9 


1.2 



Table 2. Experimental results for the Zeroconf protocol. Time in seconds. 



